Hybrid Indoor Localization via Reinforcement Learning-based Information Fusion

The paper is motivated by the importance of the Smart Cities (SC) concept for future management of global urbanization. Among all Internet of Things (IoT)-based communication technologies, Bluetooth Low Energy (BLE) plays a vital role in city-wide decision making and services. Extreme fluctuations of the Received Signal Strength Indicator (RSSI), however, prevent this technology from being a reliable solution with acceptable accuracy in the dynamic indoor tracking/localization approaches for ever-changing SC environments. The latest version of the BLE v.5.1 introduced a better possibility for tracking users by utilizing the direction finding approaches based on the Angle of Arrival (AoA), which is more reliable. There are still some fundamental issues remaining to be addressed. Existing works mainly focus on implementing stand-alone models overlooking potentials fusion strategies. The paper addresses this gap and proposes a novel Reinforcement Learning (RL)-based information fusion framework (RL-IFF) by coupling AoA with RSSI-based particle filtering and Inertial Measurement Unit (IMU)-based Pedestrian Dead Reckoning (PDR) frameworks. The proposed RL-IFF solution is evaluated through a comprehensive set of experiments illustrating superior performance compared to its counterparts

BLE-based Indoor Localization and Contact Tracing Approaches

Internet of Things (IoT) has penetrated different aspects of modern life with smart sensors being prevalent within our surrounding indoor environments. Furthermore, dependence on IoT-based Contact Tracing (CT) models has significantly increased mainly due to the COVID-19 pandemic. There is, therefore, an urgent quest to develop/design efficient, autonomous, trustworthy, and secure indoor CT solutions leveraging accurate indoor localization/tracking approaches. In this context, the first objective of this Ph.D. thesis is to enhance accuracy of Bluetooth Low Energy (BLE)-based indoor localization. BLE-based localization is typically performed based on the Received Signal Strength Indicator (RSSI). Extreme fluctuations of the RSSI occurring due to different factors such as multi-path effects and noise, however, prevent the BLE technology to be a reliable solution with acceptable accuracy for dynamic tracking/localization in indoor environments. In this regard, first, an IoT dataset is constructed based on multiple thoroughly separated indoor environments to incorporate the effects of various interferences faced in different spaces. The constructed dataset is then used to develop a Reinforcement Learning (RL)-based information fusion strategy to form a multiple-model implementation consisting of RSSI, Pedestrian dead reckoning (PDR), and Angle-of-Arrival (AoA)-based models. In the second part of the thesis, the focus is devoted to application of multi-agent Deep Neural Networks (DNN) models for indoor tracking. DNN-based approaches are, however, prone to overfitting and high sensitivity to parameter selection, which results in sample inefficiency. Moreover, data labelling is a time-consuming and costly procedure. To address these issues, we leverage Successor Representations (SR)-based techniques, which can learn the expected discounted future state occupancy, and the immediate reward of each state. A Deep Multi-Agent Successor Representation framework is proposed that can adapt quickly to the changes in a multi-agent environment faster than the Model-Free (MF) RL methods and with a lower computational cost compared to Model-Based (MB) RL algorithms. In the third part of the thesis, the developed indoor localization techniques are utilized to design a novel indoor CT solution, referred to as the Trustworthy Blockchain-enabled system for Indoor Contact Tracing (TB-ICT) framework. The TB-ICT is a fully distributed and innovative blockchain platform exploiting the proposed dynamic Proof of Work (dPoW) approach coupled with a Randomized Hash Window (W-Hash) and dynamic Proof of Credit (dPoC) mechanisms

Multi-Agent Reinforcement Learning via Adaptive Kalman Temporal Difference and Successor Representation

Development of distributed Multi-Agent Reinforcement Learning (MARL) algorithms has attracted an increasing surge of interest lately. Generally speaking, conventional Model-Based (MB) or Model-Free (MF) RL algorithms are not directly applicable to the MARL problems due to utilization of a fixed reward model for learning the underlying value function. While Deep Neural Network (DNN)-based solutions perform well, they are still prone to overfitting, high sensitivity to parameter selection, and sample inefficiency. In this paper, an adaptive Kalman Filter (KF)-based framework is introduced as an efficient alternative to address the aforementioned problems by capitalizing on unique characteristics of KF such as uncertainty modeling and online second order learning. More specifically, the paper proposes the Multi-Agent Adaptive Kalman Temporal Difference (MAK-TD) framework and its Successor Representation-based variant, referred to as the MAK-SR. The proposed MAK-TD/SR frameworks consider the continuous nature of the action-space that is associated with high dimensional multi-agent environments and exploit Kalman Temporal Difference (KTD) to address the parameter uncertainty. The proposed MAK-TD/SR frameworks are evaluated via several experiments, which are implemented through the OpenAI Gym MARL benchmarks. In these experiments, different number of agents in cooperative, competitive, and mixed (cooperative-competitive) scenarios are utilized. The experimental results illustrate superior performance of the proposed MAK-TD/SR frameworks compared to their state-of-the-art counterparts